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Abstract 

In this report, a method for exploiting the redundant degrees of 
freedom of a hand-arm mechanical system for manipulation tasks is 
illustrated. The basic idea is to try to take advantage of the intrinsic 
capabilities of the arm and hand subsystems in terms of amplitude 
of motions, different velocity limits and degrees of precision for the 
achievement of a particular task. The Jacobian transpose technique, a 
well-known algorithm for the solution of the kinematic inverse problem, 
is at the core of the proposed method for the control of the hand- 
arm system. Different behaviors of the hand and of the arm are then 
obtained by means of constraints in Null(3) added to the solution 
given by the Jacobian transpose method. The constraints are generated 
by non-orthogonal projection matrices, computed on the basis of the 
behavior desired from the system, without resorting to extended task 
space techniques. Comments about the computation of the constraints, 
and how to take advantage of them, are reported in the paper, as well 
as a description of the experimental activity currently in progress on a 
robotic system (a Puma 560 with the Salisbury Hand) at the Artificial 
Intelligence Laboratory, M.I.T. 
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1 Introduction 

In the last few years, a growing interest has arisen in the field of manipulation and artic- 
ulated hands. Major topics in this area have been the design and development of suitable 
mechanical devices, see [1, 2, 3, 4] for a few examples, and the development of techniques for 
the effective use of an articulated hand for grasping and manipulating objects, see [5, 6, 7] 
and many others. 

Recently the attention of some researchers in this field has been focused on the redun- 
dant hand-arm system, and how to deal with the whole device. The approaches taken so 
far have dealt with the satisfaction of some optimality criteria, which consider a stable and 
feasible grasp as a major goal, [8, 9]. In [8] the problem of the control of the whole device is 
separated into the two sub-problems of first choosing a suitable grasp pose for the hand, and 
then defining the arm position and orientation on this basis. In [9] the use of a non-linear 
programming technique, including several optimality criteria for the joints and the grasp, is 
proposed for the effective planning of the hand-arm trajectory. 

Since the hand-arm system can be considered a redundant manipulator, it seems natural, 
in order to exploit fully the nature of the device, to investigate the results of research in the 
area of control of redundant manipulators in order to seek techniques that can be profitably 
used in this context. 

The first observation is that the results presented in the field of redundant robots mainly 
deal with serial-link open-chain manipulators, i.e., devices constituted as a serial chain of 
links/joints. Redundancy is prevalently used for satisfying one or more secondary criteria, 
such as obstacle avoidance, singularity avoidance, optimization of task space indices or joint 
space criteria, while achieving the main task, for example the tracking of a planned trajectory 
for the end-effector. 

On the other hand, there are some particularities, characterizing a hand-arm robotic 
system, which are relevant for the development of a suitable control strategy. 

The first concerns the type of redundancy involved in the device. In [10] an interesting 
solution to the redundancy problem is presented: the proposed approach consists of con- 
sidering a redundant arm as a multi-arm system in which non- redundant arms are serially 
connected together. The task of the end-effector is consequently separated into sub-tasks 
assigned to each of the sub-parts on the basis of the individual capabilities. This method, 
although original and interesting, cannot be integrally adopted in the present context since 
the device under consideration is not a serial robot. As a matter of fact, in our case a ma- 
nipulator has a redundant parallel device -the hand- installed as an end-effector. Therefore, 
it is not possible to consider it as a serial mechanical chain. 

A second comment may be made with respect to the "behaviors" which are expected from 
this type of mechanism. In fact, usually, both small and large motions are involved, requiring 



different parts of the system to act in different ways. For example, in some circumstances 
it may not be desirable to move the arm, while in others the task may not be accomplished 
with only the limited motion capabilities of the fingers. On the other hand, the system 
cannot be generically considered as a macro/micro manipulator system [11, 12], because this 
"behavior" difference may not be needed, nor required, in other manipulation tasks. 

A third characteristic of the system being considered is that, intrinsically, a hand-arm de- 
vice has to interact physically with the environment. Therefore, any adopted control scheme 
must deal with the problem of force control along with the redundancy resolution. 

In this work a method for exploiting the redundancy of a hand-arm mechanical system 
which addresses the above mentioned considerations is presented. The proposed technique 
uses the intrinsic capabilities of the arm and hand subsystems in terms of amplitude of 
motions, velocities, and degrees of precision for the achievement of a particular task. The 
technique, which relies on a kinematic inversion algorithm for redundant manipulators well- 
known in the literature, is based on a task-space description of the task, in terms of motions 
and/or forces applied to the object/environment. The algorithm, which is presented in lit- 
erature as a closed-loop control scheme, uses as a central element the transpose J T of the 
Jacobian matrix J of the manipulator. The basic form of the algorithm is modified here, 
adding constraints on the motions of the joints in the null space of J in order to take advan- 
tage of the differencies in the capabilities of the arm and the hand. 

The paper is organized as follows. Section 2 gives a general background of the most 
popular techniques proposed in the literature for controlling a redundant manipulator. In 
Section 3 the basic scheme of the adopted kinematic inversion method is illustrated, while in 
section 4 the modifications are presented and discussed. Section 5 reports some simulations 
with a planar 4 degree-of-freedom redundant manipulator, while section 6 illustrates the 
results obtained with the implementation of the algorithm on our system, a Puma 560 
carrying a Salisbury Hand. Section 7 concludes with some comments and plans for future 
activity. 

2 Background 

The chosen approach for controlling the hand-arm system has been to consider the hand-arm 
as a redundant manipulator, and to apply, with proper modifications, techniques adopted for 
redundant robots. As is well-known, one of the major problems in this field is represented 
by the solution of the kinematic equations, since it is not possible in general to solve them 
in a closed form. 

In general, the kinematic problem can be stated as follows. Given the joint space Q, 



with dim(<5) = m, and the task space X, dim(X) = n, with n<m in the redundant case, the 
forward kinematics of a manipulator is expressed by a differentiable function: 

f : Q -* X, 
which maps each set of joint angles q in Q into the corresponding position x in X: 

x = f(q). (1) 

Accordingly, an inverse kinematic function gives a joint-space vector q for each x in the 
work-space of the manipulator, i.e. 

q = f" X (x), (2) 

such that f(q) = x. 

Often, only translational displacements are considered in X, and therefore n<3, with 
X = R n , while in the general case both translational and rotational displacements are taken 
into account, and therefore n<6, with X = R n ' U R nr , n = nj + n r , nj < 3, n T < 3. 

Certainly, the most immediate approach to the solution of the inverse kinematics prob- 
lem is to determine a closed-form solution of (2), [13], but, unfortunately, it is not possible 
to compute such a solution for every manipulator [14]. The inversion algorithms which are 
proposed in literature mostly fall into one of two major categories: (a) global (or path) 
inversion methods, and (b) local inversion methods. A global performance criterion is mini- 
mized in the first class of techniques, while locally optimal solutions are sought in the second 
one. However, the local optimization approach is the most widely adopted, since the global 
inversion methods are mainly limited to off-line trajectory planning [15, 16]. 

The local inversion methods are based on the differential relationship 

X = J(q)q (3) 

derived from (1), where q are the joint velocities, x the task space velocities, and J = Si/Sq 
is the Jacobian matrix of f (q). Eq. (3) is then usually solved using a generalized inverse, or 
the Moore-Penrose inverse [17], J + of the Jacobian matrix J. 

One of the proposed method for solving (3) consists of expressing the solution q as 

q = J(q) + x + a[I - J(q) + J(q)] VJ5T(q) (4) 
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where the first term on the right hand side of (4) is the minimum-norm least-squares so- 
lution of (3) (once an appropriate metric is defined in the Q and X spaces [18, 19]), while 
the second term represents a vector in Null(3), the null space of J, which is used for sat- 
isfying one or more secondary criteria, i.e. to optimize the differentiable objective function 
iT(q). Examples of such secondary criteria include: avoidance of obstacles, joint limits or 
singularities; optimization of task space indexes (such as manipulability ellipsoids, dexterity 
measures, ...), or joint space criteria (torque/ velocity), and several others. See [20] for a 
general overview of the most commonly adopted criteria. 

A conceptually different way for redundancy resolution has been proposed in [21] and 
more recently reformulated in [22]. In this approach, redundancy is used to accomplish an 
additional constraint task along with the original one. With this method, known as task- 
space augmentation or extended Jacobian technique, one must specify an additional task, 
expressed as a proper function of the joint variables, h(q) = 0. The solution is then com- 
puted in terms of the extended Jacobian J = [(8f/6q) T (8h/6q) T ] T . 

Another relevant approach has been proposed in [23, 24]: the task- priority-strategy. In 
this approach, a low-priority task is fulfilled in the null space of a higher priority task, solving 
in this way possible conflicts between different tasks. 

The previously reported methods are based on a generalized inverse of the Jacobian 
matrix J (or on its pseudoinverse), and give the solution in terms of the joint velocities q. 
Therefore, in order to effectively solve the inverse kinematic equation (2), one must integrate 
q to obtain the joint positions q. This is usually done directly, in an open-loop fashion, pos- 
sibly leading to non-accurate solutions q because of the linearization performed with the 
introduction of the Jacobian matrix. Moreover, since the core of this method is a differen- 
tial relationship, the resulting solution fails if appropriate initial conditions are not provided. 

A solution to these inconveniences has been found in reformulating the problem in terms 
of a closed-loop scheme [25, 26]. In [25] two closed-loop schemes are proposed for the solution 
of (3), one based on J + and the other on the transpose of the Jacobian matrix, 3 T . In [26], 
the latter scheme is independently proposed, as a general technique for solving, with some 
very general limitations, any set of nonlinear equations. This technique has been recently 
improved and extended to different cases, see [27]-[34]. Here, the main idea is to reformulate 
the inverse kinematic problem in terms of the convergence and stability of an equivalent 
closed-loop control system. This leads to the possibility of effectively solving the inverse 
kinematic problem for redundant manipulator in a robust and accurate way. 

The scheme, referred to here as the Jacobian transpose method, [34], Fig. 1, also has 
other interesting properties. A first feature is that it requires only the computation of the 
forward kinematic functions (f(q), J(q)), avoiding, in its basic formulation, the generalized 



*<*('),->«(') 


K E 




J T 




s n 




X 




/ 


9(0 


+ s 
































x(t) 


m 































Figure 1: The basic scheme of the Jacobian transpose method. 

inverse of the Jacobian matrix. This leads to a reliable scheme without numerical problems 
and instabilities such as those related to singular configurations of the manipulator. More- 
over, the stability of the scheme may be easily demonstrated using a Lyapunov analysis. 
In the continuous time domain, it can be proven that the tracking error may be arbitrarily 
reduced with an high gain A. In discrete time, a compromise has to be accepted between the 
convergence velocity and the stability of the algorithm [29, 34]. An additional interesting 
feature is that it is very simple to add constraints on the joint motions for the achievement 
of desired "behaviors" of the system. Finally, besides generating joint positions, the scheme 
provides also joint velocities q(£)> and, with minor modifications, also joint accelerations 
q(t), [26]. 

Because of the above reported considerations, it was decided, in this first stage of work 
with the hand-arm, to adopt an inversion kinematic scheme based on this technique for the 
kinematic control of the system. 



3 The basic algorithm 

The basic scheme of the inverse Jacobian algorithm, as proposed in [29, 33], is shown in 
Fig. 1. In the Figure, x<j(<) is a desired trajectory, x(t) is the actual trajectory of the ma- 
nipulator, q(f) and q(<) are the joint velocities and positions respectively, J is the Jacobian, 
K.E is a stiffness matrix, and A(> 0) is a gain which affects the convergence velocity of the 
algorithm itself. 

An interesting interpretation of the scheme, which helps to give a physical insight into 
the technique, is the following. It is well-known that the static relationship between the 
forces F applied at the end-effector and the joint torques r is given by 

r = J r F. 
The computation of the joint velocities q in the scheme, see Fig. 1, may therefore be 



related to the generation of a restoring force F = XK. E e(t) due to a positional error of an 
ideal manipulator, with null mass and unit viscous damping factor. 

In the continuous time domain, the proof of the convergence of the joint positions q(t ) 
to a set qa{t) such that f(q<*) = x,* is quite straightforward. Given the tracking error defined 
as 



e(t) = x d (t)-x(t), (5) 

and defining a Lyapunov function as 

V(e) = 5-|^ > 0, (6) 

then 



V(e) = e T Kle = e T K T E (± d - Jq) (7) 

in which the dependency of the functions from the time t and the joint positions q is omitted. 
With the choice 

*= [A+ iJSSLf *- x > °- (8) 

it is easy to see how (7) may be made negative definite. This implies that e(i) — + and 
therefore q(t) — > q<f(£). In [29] it is pointed out how eq. ( 8) may be, for computational 
convenience, simplified to 



q - AJ T K^e, (9) 

allowing the function V to be negative-definite only outside a region of the error space con- 
taining the stability point e = 0. With the choice (9), the maximum tracking error is directly 
related to x<* and inversely to A, while in steady state, since Xj = 0, the tracking error will 
be zero. In this situation, an increase in the gain A results into a reduction of the tracking 
error, which may therefore be arbitrarily reduced [28, 31]. 

A discrete time stability proof of the algorithm is presented in [34]. In this work it is 
also observed how this technique can be related to the context of non linear optimization. 



As a matter of fact, the algorithm may be interpreted as the steepest descent method, a 
well-known technique in the area of multidimensional optimization problems [35]. One of 
the major modifications in the discrete time version of the algorithm is that, in order both 
to obtain the maximum convergence rate for the scheme and to avoid instability problems, 
the gain A has to be updated at each sampling period T. In fact, given the discrete time 
version of the Lyapunov function (6) at t — nT 



and computing the joint velocities as 

q n = \ n J*K E e n , (10) 

the difference V^+i — V n may be made negative with the choice 

> _ _1_ e n Kff J n S n J w K#e n , v 

n " T e T n K T E 3 n S n 3lK E 3 n S n 3lK E e n K ] 

where S n is a diagonal matrix whose elements are properly computed to limit the maximum 
values of r, (see Fig. 1), [34]. 

A final remark on the characteristics of this scheme concerns the possibility of getting 
"stuck", i.e. to generate a null joint velocity vector, q = 0, also if e ^ 0. This happens when 
K#e £ Null(3 T ). However, this does not represent a serious limitation to the applicability of 
the algorithm. As a matter of fact, besides being an easily detectable condition (q = 0; e^ 
0), it can be argued that the term K^e can be easily modified in such a way that K^e 
Null(3 ). This is performed by adding suitable constraints on the joint space (as done in 
[34]), or in the task space. Obviously, in this latter case if the trajectory has some components 
which are constantly in Null(3 T ), the algorithm will not be able to compensate for errors in 
them. 

4 The adopted algorithm 

In order to apply the inversion technique outlined in the previous section, some modifications 
are necessary to address the particularities of the hand-arm system we are considering. The 
first problem is that the method, in its basic form, does not make any distinction among the 
various joints of the arm and of the hand. This is not acceptable, since, as mentioned in the 
Introduction, there are tasks in which a different action is desired from the two subsystems. 



For example, in order to quickly approach an object, one could take advantage of the fast 
movements of one part of the system, say the hand, while a slower motion is executed by 
the arm, restoring at the end of its motion the hand to an appropriate position for an opti- 
mal grasp of the object. In other circumstances, the motion of the hand is not needed, nor 
desired: if an object is stably grasped, it could be desirable for the arm alone to generate 
the motion of the hand/object in the environment. Therefore, in different circumstances, a 
"difference" in the "behaviors" of the two sub-systems is required. As a consequence, the 
algorithm has to be modified in order to be able to adapt the joint position/velocity values 
of the two sub-systems to the different requirements of the task being performed. Another 
capability, which may be of interest, is the possibility of maintaining some joints (for example 
the joints of the hand) both far from critical positions (singularities, joint limits) and close 
to desired ones (suitable for optimal grasp). 

In the following, let us indicate with the subscript "S" the quantities of the whole system, 
for example the Jacobian J5, the joint velocities qs, etc., while "P", "H" and "F" denote 
the arm (Puma), the hand, and a finger respectively. For the system, eq. (3) becomes now 



B x 5 = 5 J 5 qs = [ B 3p B Jh } 



qps 

4ffs 



(12) 



where the superscript "B" indicates that the quantities are expressed in the base frame B, 
Fig. 2. 

The modifications which it may be necessary to introduce into the solution given by the 
basic algorithm, shown in Fig. 1, must not interfere with the main task of the manipulator. 
This is accomplished if the modifications operate in the null space of the Jacobian J5, with 
the additional advantage that the stability proofs of the algorithm, given in the previous 
section, still apply. Hence, the final set of joint velocities q^ may be thought to be composed 
of two terms: 

qs = <U + qjv = A B 3^K E e + q N 

where qjy € Null(3s), and q^ is the solution computed by the basic Jacobian transpose 

algorithm. Obviously, since qjv = I i{p N q^ I 6 Null(3s), the following relationship 
must hold: 

= 3pi{pN + Jjyqjftv- 

In the following subsections, the determination of the term q^ is discussed, considering 
for simplicity only one finger of the hand and only linear displacements for the end effector. 
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Figure 2: The Puma - Salisbury Hand system. 



Three basic cases are considered, namely the execution of a task involving: (a) only the 
motion of the finger joints; (b) only the motion of the arm; (c) the motion of the whole 
system including, during the last part of the trajectory, the achievement of the desired 
position of the finger. 

4.1 Generating motion only with the finger. 

In this case it is required that the joints of the arm be maintained at a constant value (qp, n it), 
i.e. that qp$ = 0, and that the finger joints set-points be computed in such a way to follow 
a desired task space trajectory x<i. The final solution of (12) is then in the form 



qs = 



qps 



o 

4fs 



(13) 



with 



The joint velocities qjv may then be computed from 



(14) 



qps 





qpx + qpw = 

Jpqpiv + Jfqfn 



and, in matrix form, 



<lN2 = 



J p 3 F i\p A 

-Qfa 



B J+B 

-1/ 



j+*jp 



«u = Ppcu- 



(18) 



where Ip is the UfXUf identity matrix. The matrix Pp in (18) generates, given a solution q^, 
a joint velocity term qjv in Null(3) which, added to q^, verifies the two conditions expressed 
by (16) and (17). The matrix Pp, similarly to Pp, may be considered a projector of the 
given solution q* in the null space of Js. 

4.3 Moving the finger to a desired position. 

The actions to be determined now are intended to achieve the joint position vector qp to a 
desired value qpj, while the whole system is following a specified trajectory x^. A practical 
way for the achievement of this goal is to generate a joint velocity term which compensates 
for the positional errors (qpj — qp). In other words, the term qw may now be computed 
from 



= Jpqpjv + Jpqpjv 
qpw = K(qp<i — qp) 

from which 

qpjv = — J P JpK(qpd — qp) 
qpjv = K(qp d — qp) 

or, in matrix form, 



qjv3 = - 



B 3 P B 3 F 
-I, 



K(qp«i - qp) = -PpK(qp,i - qp) 



(19) 



in which the projector Pp is the same as in (18). 



The three terms qjv.' in eq. (15), (18), (19), may be combined together, resulting in the 
scheme shown in Fig. 3. In the scheme, the three factors ai, a 2 , and 0:3 (0 < c*j < 1, 
i=l,2,3), are variable gains which are used to modulate the actions of the three terms qjvx, 
qjV2> and qjv 3 on the original solution q^. By properly changing in real time the three factors 
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x d (t) e(t) 



Figure 3: The modified Jacobian transpose scheme. 

aj, it is possible to modulate the possible "behaviors" of the system. If cq = 0, the relative 
constraint term q#; is neglected, while when c*i = 1, the constraint is fully active. 

A final remark may be made with respect to the projectors Pp and Pj>. As a matter 
of fact in these matrices a term of the type J^ Jjg is present (the sub-matrix B 3p B 3p in 
Pjr or the sub-matrix B 3~p B 3p in Pp). In the two cases, this implies that the original 
trajectory is tracked without errors iff 3b<Ib € Range^J^). If Range(3 a) = Range(3s), the 
introduction of the projectors does not change the trajectory of the system, otherwise, only 
the component in Range{3 a) may be compensated for with this solution. If only one finger 
is considered, this problem may become relevant only with the use of the matrix Pjr, since 
(with the exceptions of the singularities) the range space of the arm is the whole % 6 . 

4.4 Computation of the gains a,-. 

An interesting problem is how to define suitable strategies for the computation in real-time 
of the scalars o^. The solution qs is now expressed as 

qs = <U + «iqjvi + c*2qjV2 + «3qjV3- 

It is not meaningful to have aj = a 2 = a 3 — 1, since this would imply the simultaneous 
activation of opposing constrains on the solution, while the choice a t = a 2 = a 3 — means 
that the original solution q^ is adopted. In particular, the choice a x = a 2 = 1 implies 
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that the two contradictory constraints of keeping both the arm and the hand blocked while 
following a trajectory are imposed. As a matter of fact, this choice results in a 'switching' 
of the motions produced in the end-effector by the the two subsystems, and the final effect 
is simply to have a different set of joint velocities q$. 

As previously mentioned, it may be convenient to take advantage, at least during the 
first period of motion, of the fastest part of the system. It seems therefore reasonable to 
have, at the beginning of a task, the gains set as 

a x = 1, a 2 = a 3 = 0. 

These values have to be changed when, for example, the finger's joints are close to a limit, 
or when the tracking error ||e|| is greater than a maximum allowed value ||e|| ma:i .. A way to 
take these constraints into consideration is to compute the gains as 

/ 1 if -q T i <qi< qn,i = l,...,»/5 and ll e ll < ll e llma* / 9fl N 

«i - | e -„A,-„A. otherwise; W 

a 2 = (l- ai )0 (21) 

a 3 = (l-ai)7 (22) 

where n/ is the number of joints of the finger, qxi a threshold value which limits the motion 
of the i-th joint , and qu the actual joint limit, Aq = mat f ^ , Ae = ||e|| — ||e maa; ||, fi,u 
two positive scalars which are set to 1 if the relative limit is broken. In this way the finger 
takes care of the required motion, provided that all the joints, as well as the tracking error, 
are within proper thresholds. If one of these two conditions fails, with the choices (20)-(22) 
the arm starts moving, while the finger achieves the desired configuration. 

If other "behaviors" of the system are desired, different combinations of the three gains 
are possible. For example, if an object has been grasped, and there is no need of modifying 
the grasp pose, a 2 is set to 1, and all the movements of the system are generated by the arm. 

4.5 The force feedback loop. 

The final aspect of the problem we deal with is the need to consider, in the generation of the 
joint trajectories, the forces that are applied to the environment. To this purpose, a further 
feedback loop is added to the scheme, leading to the diagram shown in Fig. 4. In this 
scheme, F<j is a desired force, ep = (Fj — F) is the force error, and Kp a compliance matrix 
which represents a model of the manipulator-environment interaction. The stability of this 
loop is now affected by the manipulator dynamics and its control system. Considering an 
ideal system, i.e. assuming the forward kinematics f(q) as model of the manipulator/control 
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Figure 4: The force feedback loop in the Jacobian transpose scheme. 

system, and a stiffness matrix Kjr for modeling the interaction with the environment, the 
stability proof follows the same line as before. This model will be used in the following 
discussions. 

4.6 Extension to the whole hand. 

When the whole hand is taken into consideration, it is necessary to consider an extended 
task space. In fact, we are interested now not only in the specification of the object posi- 
tion/orientation, but also, once the object is grasped, in the relative displacements of the 
fingers, i.e. in the distances between the fingertips. Considering three fingers, the new 
forward kinematic function is therefore 



«(q) = 



f(q) 

d(q) 



en* 



(23) 



where f(q) 6 H 6 are the forward kinematic equations relating the hand-arm joints values 
to the position/orientation of the object, and d(q) £ H 3 are the relative displacements of 
the fingertips. It is easy to see how only the joints of the hand affect this latter vector. 
Assuming a grasp on a rigid object, and without slip at the contact points, the computation 
of these two functions is quite straightforward. In particular, the function d(q) results as 
the magnitude of the difference of the positional vectors of the three fingertips, and, how 
already pointed out, is a function only of the hand joints. We are now interested in defining 
a differential relationship between the set of joint velocities and the set of object velocities 
and "internal velocities", i.e. the relative velocities of the fingertips. 
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In [1] the notion of grasp matrix G has been introduced. This matrix is a 9x9 matrix 
which relates the forces and displacements of the object to the forces and displacements of 
the single fingers, i.e. 



f = g t :f 



or, in the velocity domain, 



■kf = G 



-l. 



(24) 



where 
F = 

T = 

X/ = 



Y\ F^ Fj represents the forces applied at the fingertips 

f r t T /12 /23 /31 I are the 6 external and 3 internal forces acting on the object 
x^j xj 2 xj 3 are the three linear velocities of the fingertips 

-iT 

di2 d,23 d 3 i I are the 6 velocities of the object and the relative velocities 



rl 



of the three contact points, expressed as function of the hand joints only. 
Eq. 



(24) may be written as 

H J 2 q 2 
H J 3 q 3 



H 



X/ = 



H 3 1 
H 3 2 
H 3 3 



then 



x = c=r j h <\h = Jh<Ih = 



qi 

q 2 

qs 



H 3 



Hfi 



J»qn = G 



-1 H, 



Bf 

H 3 H a 



q» 



(25) 



in which all the quantities are expressed in the hand frame %, see Fig. 2, and the two 

terms H 3uf = I H ^ v H ^Hu> J an( i H ^Bd refer respectively to the object velocity and to 
the "internal velocity", i.e. the deformations of the grasp triangle, the imaginary triangle 
between the three contact points. Equation (25) may be expressed in the base frame as 



x„ = 



*Vff *Vff -TO® 








B R 



H 








H 



H 3 



nqn 



(26) 



in which R# is the 3x3 rotational matrix expressing the rotation between the frames H 
and B, and Po® is a skew symmetric matrix equivalent to the cross product (x B p &j), where 
the vector B Pobj gives the object position in the base frame. 
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As far as the arm is concerned, a relation similar to (26) may be denned. The derivatives 
of the functions f (q) and d(q) with respect to the arm joints yield the Jacobian for the arm: 



B f — 
Jp = 


' 6f/8qp ' 




Jpf 


Sd/Sqp 







where the matrix B 3p/ computes as follows 


Jp/ = 


' ( B Jpv - PX J 

Bl 

Jpo, 


3 Jpo>) ' 



where B 3pf = B Jp„ B Jp« * s * ne arm Jacobian, in which the terms generating the 
linear and rotational velocities are in evidence. The matrix Px is a skew symmetric matrix 
equivalent to the cross product (x B p ), denned on the basis of the elements of the vector 
B p , giving the object position with respect to the hand frame, expressed in the base frame. 
Therefore, the final Jacobian 3$ of the hand-arm system, in our case a 9x15 matrix, is 



>3 s =[ b Jp B 3 H ] = 



B 3 Pf B 3 Hf 



B 3 



Hd 



and the differential relationship between the set of object velocities and internal velocities 
and the set of joint velocities is 



x = 



3pf 3jjf 
3 Hd 



qp 



= 3 s i\s 



where the superscript B is omitted for brevity. 



The Jacobian transpose method may still be applied, with the additional need of speci- 
fying the internal displacements d. This implies that the solution q^ given by the algorithm 



is 



qp = \3 pJS-Ef^f 

i\jj = X(3 Hf K E fef + 3 Hd K Ed e d ) 

from which it is clear how the joints of the Puma may compensate only for errors in the 
position/orientation of the objects, while the joints of the fingers also affects the internal 
velocities. In this case, without the introduction of the projectors, if d(t) is maintained at 
a constant value and a movement of the object is required, both qp and q# are changed in 
order to follow the trajectory, with the additional requirement for the hand to maintain a 
constant grasp triangle. 

In the following, the three cases of movements of the hand, of the arm, or changes in the 
relative displacements of the fingertips are considered. 
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4.6.1 Case 1: motion of an object grasped in the hand with only the joints of 
the fingers. 

If a modification of the grasp triangle is not required at the task level, i.e. d=cost., the 
specified trajectory originates a velocity vector of the form io= Xy I , therefore 



x / 




Js<is- 



If it is desired to move the grasped object by using only the hand, it follows that 

qps = 4pa + qpiv = o 

Jsq* = 



therefore 



from which 



-JpfClPA + Jtf/qflTV = 
J/r<*q.ffjV = 



<lHN = ^Hf^Pf^PA 



or 



qjv 



-qj>A 



J+ y J P/ 



cu = Ptfqx, 



which is equivalent to eq. (15) obtained in case of a single finger. 

4.6.2 Case 2: motion of an object grasped in the hand with only the joints of 
the arm. 

At the task level, we assume now that the same trajectory of the previous case is specified. 
It is now required that 



x / 




Jsqs 



with 



17 



Jsqjv = o 



therefore 



Jp/QPAT — Jff/qtfA = 
— ^Hdi{BA = 0. 



Hence, one computes 





qpff = 3 p 3h<Iha 


or 

qj\r = 


B n B 3 H iiHA' 

—i\HA 


= 


B 3+ B 3 H 
-I, 


equivalent to eq. (18) ob 


tained previous] 


y- 





i{A = PpqA, 



4.6.3 Case 3: deformation of the grasp triangle. 

Let us consider now the internal motions only, which might be required to adjust the grasping 
forces. The desired trajectory in the task space implies that the task velocity vector is in 
the form: 




d 



= 3 s qs 



i.e. 



= Jpfilp + Jff/qjj 
d = J/jdqjf . 

It is clear that it is not possible to achieve this result while keeping the joints of the hand 
blocked (q# = =>■ d = 0). In fact, the application of the projector Pp to the solution q^ 
computed in this case by the algorithm, gives as result 



qs = <U + P P <u = 



qpA + 3p3 H qff A 
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Figure 5: The simulated 4 degree-of- freedom redundant manipulator. 



or, in terms of the final task space velocity, 

Jsqs = 



3pf 








qpA + JpJnqHx 




Jp/qpA + Jp/JpJnqHA 




and therefore no internal motions are accomplished. Only the motion of the object which 
should be generated by q# are compensated for: nothing can be done for d. In this case, 
only the projector P# could be applied without errors in the task space. 

5 Simulation 

In order to test the effectiveness of the proposed kinematic inversion technique, a simulation 
has been carried out with a planar redundant manipulator. The simulated robot is shown 
in Fig. 5. It has 4 degrees of freedom and it is basically constructed as two two-link 
manipulators emulating a planar two degrees of freedom arm carrying a two degrees of 
freedom finger. In Figs. 6-8 some results are reported. 

In particular, in Figs. 6, 7 a trajectory of the end effector in the task space and the 
corresponding trajectories in the joint space, obtained from the algorithm of Fig. 1, are 
shown. 
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Figure 8: The joint-space trajectories with the modified technique. 



Then, the same task has been performed using the algorithm described in the previous 
section. In Fig. 8 the obtained joint trajectories are shown. It may be noticed how only the 
joints 3, 4 are activated at the beginning of the task, while joints 1 and 2 start moving only 
when one of the condition in (20) fails. After that, joints 3 and 4 are restored to the original 
position, while only the first two joint actually move the end-effector along the cartesian 
trajectory. The tracking errors obtained with the original and the modified scheme are of 
the same order of magnitude (< 2 mm). 

Figs. 9, 10, show a task in which the manipulator is required to apply a force. There 
are no motion specifications at the task level: only the requirement to exert a force along 
the negative x direction; a rigid surface is positioned at x = 1.05. Fig. 9 reports the joint 
position values. Again, joints 3 and 4 start the motion, and when one of them reaches the 
joint limit, the arm begins to move until contact with the surface is detected. Finally, during 
the force application phase, joints 3 and 4 are restored to the desired initial position. In Fig. 
10 the desired and applied forces are shown. 

6 Implementation 

A first set of experiments of the above described technique has been carried out on an ex- 
perimental set-up at the Artificial Intelligence Laboratory, M.I.T. The manipulator consists 
of a Puma 560 with the Salisbury Hand installed. The system has 15 degrees of freedom: 
6 in the arm and 9 in the three fingers. Force information is available from force-sensors 
installed in the fingertips and from a sensorized palm. The control is performed by a two 
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Trajectories of Puma joints 1-3 

Figure 11: The trajectories of joints 1-3 of the Puma. 



level control system: a VMEbus with three MC68030 processors running the VxWorks real 
time operating system is used as a supervisor for the servo level, based on three Unimation 
controllers, one for the Puma and two for the Hand. Two of the MC68030's are used to 
manage the communication with the controllers of the Puma and the Hand, as well as to 
acquire and process the force information from the fingertips. The third processor is used 
for the solution of the kinematic equations. 

At the current stage of implementation, only one finger of the hand is taken into consid- 
eration in the kinematic inversion algorithm. 

In Figs. 11-13, some results obtained from the experimental equivalent tasks of Figs. 6-10 
are presented. Specifically, Figs. 11-12 show the Puma joint trajectories, and Fig. 13 the 
finger joint positions for a straight-line motion of the end effector. The modified algorithm 
is used in this case, leading to results similar to those presented in Fig. 8: the Puma's joints 
are blocked during the first period of motion, while in the last part the finger is restored to 
the initial position and the task is accomplished by the Puma. 
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Trajectories of Puma joints 4-6 

Figure 12: The trajectories of joints 4-6 of the Puma. 
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Figure 13: The trajectories of the finger's joints. 
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7 Conclusions and future work 

In this report, work aimed at the development of a kinematic inversion algorithm for an hand- 
arm system has been described. The chosen approach for the determination of such algorithm 
has been to consider the device as a redundant manipulator, and to apply, with proper 
modifications, one of the most promising techniques in the field: the Jacobian transpose 
method. The modifications introduced in the scheme consider the different capabilities 
of the device in terms of maximum joint speed and/or amplitude of motion, as well as the 
possibility of executing the specified task with only a subset of the available joints. Moreover, 
a force feedback loop has been introduced, since the application of force on the environment 
is a major goal in the tasks for the system we consider. 

At the present, only a partial implementation of the described algorithm, considering the 
arm and one finger, has been realized on an experimental set-up available at the Artificial 
Intelligence Laboratory, M.I.T., a Puma 560 with the Salisbury Hand. 

The first comment on the currently realized algorithm concerns the rules adopted for 
the computation in real time of the gains c^ in (20)-(22). In fact, these rules take into 
consideration only static or first-order kinematic constraints, such as the joint-limits or the 
tracking errors. It could be of interest to take into consideration different and more general 
rules, based also on the effective dynamic capabilities of the individual joints. 

Another interesting variation that could be introduced is to conceptually consider the 
wrist as a part of the hand rather than as a part of the arm. In this case, the "arm" would 
have only the responsibility to position the "hand" in the work-space, while all the remaining 
actions would be executed by the "hand". This should result in a more "anthropomorphic" 
behavior of the whole device, requiring no motion of the arm in manipulation tasks in which 
only small motions are required. 

Finally, it is in the authors' opinion that the performances of the algorithm could be 
improved by considering the gain A of the loop not simply as a scalar, but as a full njxrij 
matrix. As a matter of fact, when the forward kinematics function is not dimensionally 
homogeneous, some limitations of the performances, in terms of convergence of the algorithm 
to the solution, are noticed. 

These modifications of the basic algorithm, along with the full implementation of the 
proposed technique on the hand-arm system, are among the main goals of the current activ- 
ity. 
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